Software testing plays an important role in the development of computer software and is used to confirm whether or not the quality or performance of a software program conforms to some requirements raised before the development of the software. Software testing is an inspection of software requirement analysis, design specification description and coding before software is put into practice and is a key step for guaranteeing software quality. Software testing is a process of executing a program in order to find errors. Software testing may be divided into unit testing and integration testing, wherein unit testing is a testing of the minimum unit of software design, i.e., a module, while integration testing is a testing of the whole software system. After respective modules having passed unit testing are assembled together according to design requirements, integration testing is performed so as to find various interface-related errors.
Today, service-oriented software development has two important features: distributed, and community-based. This makes it necessary to perform simulation of real services whose development has not been completed yet. Considering a typical scenario in service unit & integration testing, in which some services are not implemented yet, or although implemented, they reside in a remote site and thus cannot be easily configured & used locally. In this case, it is necessary to perform simulation of these real services.
Service simulation includes at least the following two aspects: functional and non-functional. In the functional aspect, it is needed to simulate responses of real services to different requests, i.e., to simulate whether these services are capable of implementing or accomplishing predetermined functions or not. In the non-functional aspect, it is needed to simulate properties of these services, such as performance, availability, security, response time and so on.
Here, it is noted that the services in this document may indicate modules, processes and the like having predetermined functions in the software development.
The key of the service simulation is to provide a quasi-realistic execution environment for the real services under test (either unit or integration test). Since each service needs to deal with various usage scenarios (which include at least normal and exception conditions) to test these different scenarios, the simulated services should be flexible for implementing changes in behavior (i.e., have versatility) for simulating various different scenarios that can occur in the real world.
The following Table 1 lists common simulation methods used in conformity with the prior art in the real software testing (following Object-Oriented terms).
TABLE 1MethodFeaturesHow to use in testingStubStatic, simplified implementations ofTest case is separatereal classes, stored in a file system,from Stub, and testreturning a specific response to acase uses Stub like realspecific request on invocation.classes.Stub can invoke methods of otherTo simulate differentclassesbehavior of an object,Test logic (invocation sequences &multiple Stubs areinput-output mapping) is hard-codedneeded, or it is neededin Stub implementation & test cases.to modify these Stubsfor different test cases.MockIts behavior can be specifiedTest cases dynamicallyprogrammatically, including thecreate and use Mockinput-output mapping & invocationobjects.sequencing.Mock provides a built-in tool to verifythe invocation sequence &input-output (mapping) as previouslyset in a test case.A Mock object cannot invokemethods of other objects.
“Stub” is used to denote temporary and simple program codes that simulate functions of more complex codes. Stubs and real implementations have the same method definition so that method callers can use both in the same way. Here an example of a real class “Collaborator” and its Stub can be found from http://blog.interface21.com/main/2007/01/15/unit-testing-with-stubs-and-mocks/. Stub is a simplified and partial implementation of real code logic. Usually, it only describes output messages for specific input messages.
“Mock” (see http://www.easymock.org/) is used to denote a special kind of simulation mechanism using predefined application programming interfaces (APIs). Firstly, Mock objects are created at runtime, as is their behavior (what response is produced for what request). Secondly, Mock provides a built-in mechanism for test verification.
More details regarding unit testing of Mock and Stub can also be found by referring to http://blog.interface21.com/main/2007/01/15/unit-testing-with-stubs-and-mocks.
Simply speaking, the key difference between Mock and Stub is that Mock predefines a verification (assertion) tool that makes test logic writing easier than Stub where verification logic has to be hard-coded.
However, both Stub & Mock are not versatile in their behavior. Stub is a simplified implementation of real objects, and not versatile enough in its behavior. Complex test logic usually requires a lot of Stubs and a lot of simulation codes, and is difficult to manage. The behavior of Mock in a specific test case is also fixed, representing a specific test sequence.
In addition, with both Stub and Mock, it is not possible to easily simulate the non-functional aspect of the services. In other words, they are not designed to deal with non-functional simulation.
Furthermore, Mock has the following disadvantages: a Mock object cannot invoke methods outside, making it not suitable to be used in integration testing that requires a Mock object to initiate invocation of other objects.